As the number of active web sites on the internet grows, there is an increased demand for organizing and assessing the available content. Search engines have made available information easier for individual users to find and access. Typical search engines may include algorithms that aid in identifying relevant content based on a user's search terms. These may include, for example, taking into account an apparent popularity, based on web traffic, of a particular page with the requested content. However, there are limitations on the capabilities of conventional services that typically download content from a web site, index it, then allow users to search based on the downloaded content.
Understanding the ways in which the Internet is intrinsically organized can be helpful in understanding the challenges related to efficiently utilizing the full scope of information that is accessible via the Internet. The Domain Name System (DNS) is the part of the Internet infrastructure that translates human-readable domain names into the Internet Protocol (IP) numbers needed to establish TCP/IP communication over the Internet. That is, DNS allows users to refer to web sites, and other resources, using easier to remember domain names, such as “www.example.com”, rather than the numeric IP addresses, such as “123.4.56.78”, assigned to computers on the Internet. Each domain name is made up of a series of character strings (labels) separated by dots. The right-most label in a domain name is known as the “top-level domain” (TLD). Examples of well-known TLDs are “.com”; “.net”; “.org.” etc. Each TLD supports second-level domains, listed immediately to the left of the TLD, e.g. the “example” level in “www.example.com”. Each second-level domain can include a number of third-level domains located immediately to the left of the second-level domain, e.g. the “www” level in “www.example.com”. There can be additional level domains as well, with virtually no limitation. For example, a domain with additional domain levels could be “www.photos.example.com”.
It should also be noted that a single IP address, e.g. a single server, can support numerous domain names. That is, different domain names may resolve to the same server, that can then determine what content to provide based on the requested domain name and/or additional non-domain information. This is sometimes referred to as virtual hosting.
Additional non-domain information may be included in a Uniform Resource Identifier (“URI”) structure that includes the domain name. For example, a “path” part is a sequence of segments (conceptually similar to directories, though not necessarily representing them) separated by a forward slash (“/”). This information may be included immediately to the right of the domain name, such as the “blog” in “www.example.com/blog”, and may be used by a server or other receiving device to identify and deliver specific content or run particular code. Other examples of non-domain information may include queries and fragments, the specifics of which are understood by those of ordinary skill in the art and are not discussed in detail herein. Combinations of this information may be included in web page hyperlinks that navigate a user to another section of the same page or to another web page that may be part of the same, or a different, domain.
Related domain names, and content, may be organized in a hierarchical, or nested, manner, such as “www.example.com”; “www.blog.example.com”; “www.example.com/blog”; or “blog.example.com” etc, each with a different significance. Such related domains need not share similarities in the actual IP address to which the various domain names resolve to. In this regard, part of the domain name may signify a particular server which is desired, for example, “mail.example.com” and www.example.com” may resolve to different servers, with different functions, for the same second-level domain.
The responsibility for operating each TLD (including maintaining a registry of the second-level domains within the TLD) is delegated to a particular organization, known as a domain name registry (“registry”). The registry is primarily responsible for converting domain names to IP addresses (“resolving”) through DNS servers that maintain such information in large databases, and operating its top-level domain.